The present invention relates to a non-linear color corrector and more particularly, to a non-linear color corrector which includes a neural network for defining the class of colors to be changed.
There are many possible technical problems in image reproduction, including the effects caused by lighting and non-ideal image sensing, and transmission and display of the image. Non-linear processes can occur at any one of these steps to corrupt the color fidelity of the image.
For example, problems may arise in the video camera used to sense an image. A video camera is made up of three separate channels, usually identified as the red, green, and blue primaries. The required spectral sensitivity of the three camera primaries can be derived using the chromaticities, a physical measurement related to spectral content, of the display device's three primaries. The camera primaries are related to the spectral sensitivity of the prism that separates the light into three channels. Color reproduction fidelity suffers to the extent the camera primaries do not match the three primaries of the display.
There is also the possibility of introducing a simple linear transformation into the three primaries, sometimes necessary when changing from one set of display primaries to another. Linear transformations are also used when it is necessary to increase the saturation of colors, or to change the brightness.
In addition to technical reproduction of an image, psychological processing of an image by a viewer effectively makes a reproduced image appear different than if the viewer were present at the scene. These effects include brightness and color adaptation. Adaptation is the tendency of the eye to adjust itself to the spectral distribution of the average illumination over the entire field of view. Thus, someone who perceives an original scene with a particular average spectral illumination would perceive the scene differently on a smaller display in a setting with a different average illumination.
In addition to the above technical problems of reproducing and displaying an image, it may be desirable to perform artistic color correction of an image. A person may simply desire to change the color content of an image for purely artistic reasons. For example, a specific class of colors in an image may be changed to a different color for purely artistic reasons. These types of corrections are non-linear transformations of a specific class of colors.
Non-linear color correction involves several steps including: (1) defining a class of colors that need to be changed (hereinafter referred to as the "change-class"), (2) defining how a pixel belonging to that class is to be changed, (3) examining each pixel and determining the extent to which it belongs to the change-class, and (4) changing each pixel according to its membership in the change-class.
The change-class can be defined by user examples. For example, the user can examine the image and generate a list of pixels that should undergo significant change, should undergo little change, or should not be changed. In addition, the user can also control how the pixels should be changed. This is a simple, model-free method of acquiring the necessary information from the user.
FIG. 1 illustrates the difficulties which may be encountered in defining the change-class using a model-free class membership definition. FIG. 1 is a graph showing a mapping of color components in a two-dimensional color image into two color components. In FIG. 1 the user's change-class is not a simple shape like an ellipse or a rectangle. As a result, it is difficult to model the shape of this class. The difficulty increases in three dimensions (i.e., when a third color component is used in the mapping operation).
The pixels in the change-class are changed once the membership in the change-class is determined. The pixels can be changed using both modelled and model-free methods.
FIG. 2 illustrates the modelled method for changing a pixel. In FIG. 2, the pixels having color combinations near pixel p1, for example pixel p3, are changed towards pixel p2 to either, for example, pixels pa or pb, having color combinations near pixel p2. A color corrector examines pixel p3, determines if it belongs to the change-class, and provides a membership value c designating the pixels membership in the change-class. The pixel p3 can be changed towards p2 to pixel pa by an amount proportional to the membership value c in the direction of vector A according to equation (1) below. EQU pa=p3+c*A (1)
Alternatively, pixel p3 can be changed in the same direction as the vector V to color combination pb according to equation (2) below. EQU pb=p3+c*V (2)
As shown in FIG. 2, the vector B=c*V.
Equations (1) and (2) essentially move a pixel along a vector by an amount proportional to its membership value c in the pixel change-class. Equation (1) moves the pixel towards the "target" pixel and equation (2) moves the pixel in the same direction as the original vector. Equation (2) maintains color resolution near the colors that will be changed by remapping pixels in the same direction. Equation (1) allows a relatively larger color-volume of pixels to be mapped to a smaller color-volume, thus, decreasing color resolution.
The implementation of either equation (1) or (2) for remapping the pixels in the change class, however, requires that the pixels in the change class be defined. Defining the change class is a difficult task requiring extensive user involvement. Defining the change class is even more difficult where, as shown in FIG. 1, the change class is not a simple shape. The class of colors to be changed can be somewhat arbitrary, fuzzy, and complex. Furthermore, some non-linear color correctors are difficult to use, requiring the user to have extensive knowledge of colorimetry principles. Thus, it would be advantageous if the user could define the class by example without identifying large number of pixels.
Accordingly, an apparatus and method are needed to define the change-class and to remap the pixels in the change-class.